In this paper, we address the challenges of energy and throughput management in a Wireless Body Area Network (WBAN) with a focus on a heart rate sensor. Our approach utilizes the sleep and wake-up method to minimize sensor energy consumption while harnessing Radio Frequency (RF) waves and human activities (running, walking, and relaxing) as Energy Harvesting (EH) sources to supplement battery power. Bluetooth Low Energy 5 (BLE5) technology is employed for wireless information and energy transfer. Our goal is to strike a balance between throughput and battery residual energy. The advantages of using 𝒬-learning for action selection in comparison to Random Action (RA) selection are demonstrated through simulations. The results reveal that the reward function in 𝒬-learning, incorporating a balancing parameter, effectively achieves a compromise between throughput and battery residual energy. Additionally, our 𝒬-learning method improves system throughput by 43% compared to RA selection. In addition, we compare the performance of the 𝒬-learning and State- Action- Reward- State- Action (SARSA) algorithms using the same reward function to evaluate their respective abilities in managing system throughput and battery residual energy. These findings have significant implications for developing energy-efficient WBANs, enabling prolonged operation and reliable data transmission.